Pennochio - Mapping Motion Capture onto a Humanoid Robot in Real Time

The objective of this project is to use motion capture to track the motions of a human subject, above the waist, and then map these motions onto a humanoid robot in real-time, essentially making the robot a puppet and the human the puppeteer. This puppeteer will control the robot while wearing a virtual reality rig that allows the puppeteer to see what the robot sees. The robot that will be used is the PR2 from Willow Garage [3] and the motion capture system was initially designed to be the VICON system [9]. However, given possible difficulties in usefulness of a system based on such an expensive system with such limited access, thought has been given to developing the system (either exclusively or in parallel) for use with more commercial motion capture systems like the Sony Playstation Move and the Microsoft Xbox Kinect. Through the course of the project the Kinect has revealed itself to be the more logical choice and so has replaced the VICON system as the method for motion capture. The PR2 can raise and lower its height, turn and tilt its head, and has arms that fully mimic a human arm. Because most motion capture relies on manual correction after gathering data, the most difficult task will be forming an algorithm or method for motion capture that avoids ambiguity among the capture points and can also be easily be mapped onto the robot in real-time without significant delay. Although the main objective is to achieve an effective mapping from the user’s head, torso and arms onto the robot in, it is also hoped that an effective system can be designed to control the robot’s motion in the (x,y) plane and also to eventually design a system using a mechanical device to grasp and interact with objects.